Market Matching

Overview

Market Matching: Overview

What is Market Matching?

Market Matching is a method used to find the most comparable control markets for a chosen test market.

It answers: "Which untreated markets behave most similarly to my test market before intervention?"

This is a critical setup step for causal measurement workflows such as GeoLift.

---

Why Market Matching matters

Good matching improves decision confidence by reducing bias in test-vs-control comparisons.

Without strong matches:

• impact can be overstated or understated,
• significance can become unstable,
• and follow-up decisions become riskier.

---

Core capabilities

1) Intelligent matching

• Automatically analyzes pre-period market behavior
• Ranks potential controls by similarity
• Supports multiple matching methods (balanced, causal, quick)

2) Quality and balance checks

• Computes similarity scores per control market
• Evaluates balance statistics (including SMD-style diagnostics)
• Produces quality scores and balance pass/fail indicators

3) Guided, review-ready outputs

• Primary matching table (test market -> ranked controls)
• Quality assessment summary
• Similarity chart and treatment-level drilldown
• PDF report artifact

---

What data it uses

Market Matching works on time-series data with:

• a date column,
• a market/location column,
• and an outcome metric column.

The pipeline auto-detects these fields and can also take user overrides.

---

Typical use cases

• Choose controls for causal testing
• Validate whether existing test/control pairs are reliable
• Compare markets for rollout planning
• Identify where intervention learning can be transferred safely

---

Typical decisions it supports

• "Are these controls good enough to trust causal results?"
• "Should we change test markets before running GeoLift?"
• "Which markets are comparable for pilot expansion?"
• "Do we need to rerun with stricter quality thresholds?"

---

Where it fits in the product experience

Market Matching is a structured workflow:

• submit job,
• run in background,
• review ranked controls and quality diagnostics,
• export report artifacts.

---

Quick value summary

Market Matching provides:

• stronger control selection,
• transparent quality diagnostics,
• and more reliable foundations for downstream causal decisions.

---

Related documentation

• Methodology — how the pipeline works technically (market-matching-methodology)
• Models — balanced, causal, quick matching explained (market-matching-models)
• Workflow — step-by-step process (market-matching-workflow)
• Results — how to read and interpret outputs (market-matching-results)

Models

Market Matching: Models

Overview

Market Matching offers three matching algorithms. All produce ranked control markets with similarity scores and balance diagnostics, but they differ in which features they use and how similarity is computed.

Method	Features used	Similarity approach	Best for
balanced	8 fingerprint features	Euclidean distance (scaled)	Default — full behavioral comparison
causal	7 features (excludes outcome_cv)	Propensity score distance	Causal-style pairing emphasis
quick	3 features (mean, std, trend)	Euclidean on core features	Fast screening or large datasets

All three use the same SMD-based balance check downstream.

---

Geo fingerprints (input to all methods)

Before matching, each market is summarized into a fingerprint from its outcome time series:

• Level — mean outcome
• Volatility — standard deviation, coefficient of variation
• Trend — Theil-Sen slope (robust to outliers)
• Seasonality — STL decomposition strength
• Stability — autocorrelation, residual variance, spikiness
• Range — min–max spread

Similarity is computed by comparing these fingerprints between the test market and each candidate control.

---

1) Balanced matching (default)

How it works

• Uses 8 fingerprint features (all except propensity-related fields)
• Features are standardized (zero mean, unit variance)
• Similarity = 1 / (1 + euclidean_distance) between test and control fingerprints
• Higher score = closer match

Strengths

• Most comprehensive behavioral comparison
• Good default for most use cases
• Balances depth and interpretability

When to use

• Standard matching runs
• When you want the fullest pre-period comparison
• When match quality matters more than speed

Plain-language summary

"Find control markets whose sales patterns look most like the test market across level, trend, seasonality, and stability."

---

2) Causal matching

How it works

• Uses 7 fingerprint features (excludes outcome_cv)
• Computes a propensity score for each market via logistic regression
• Similarity = 1 / (1 + propensity_score_distance)
• Emphasizes markets that would be assigned similar treatment probability in a causal framework

Strengths

• Aligns with causal inference best practices
• Useful when you want propensity-aware pairing
• Reduces reliance on raw outcome level alone

When to use

• When causal rigor in control selection is a priority
• When outcome level differences are large but behavioral patterns are similar
• As an alternative to balanced when results need comparison

Plain-language summary

"Match markets that are statistically likely to behave the same way, not just markets with similar average sales."

---

3) Quick matching

How it works

• Uses only 3 core features: mean_outcome, std_outcome, trend
• Same euclidean similarity formula as balanced, but on fewer dimensions
• Faster execution with simpler comparison

Strengths

• Fastest matching option
• Good for initial screening
• Works when data history is limited

When to use

• Quick exploratory matching
• Large numbers of candidate markets
• When you need a fast first pass before a deeper balanced run

Plain-language summary

"Quickly find markets with similar average sales, volatility, and trend direction."

---

Balance check (shared across all methods)

After controls are ranked, each test–control pair is evaluated for balance:

• SMD (Standardized Mean Difference) is computed per feature
• Time-series autocorrelation is accounted for in effective sample size
• Pair is balanced when all feature SMDs are within threshold

Balance status appears in:

• primary results table,
• quality assessment,
• PDF report heatmap.

---

How to choose a method

Situation	Recommended method
Standard production run	balanced
Causal experiment setup	causal
Fast screening / many geos	quick
Unsure which to use	Start with balanced, compare with causal if needed

---

Quality gate

An autonomous quality filter runs after matching:

• Pairs below minimum match quality may be dropped
• Unbalanced pairs can be excluded (configurable)
• If all pairs fail, the system may salvage the top available pair with a warning

Warning comments you may see:

• "No similar markets found for selected test market(s)"
• "Filtered out poor-quality market candidates"
• "All markets failed strict quality threshold — salvaged top candidate"

---

Key outputs per method

Regardless of which algorithm runs, each match result includes:

Output	What it means
treatment_geo	Test market analyzed
control_geos	Selected control markets
similarities	Similarity score per control (0–1, higher is better)
balance_stats	Per-feature SMD values
is_balanced	Whether the pair passes balance threshold
method	Which algorithm produced this result
quality_score	Overall match reliability (0–1)

---

For the full pipeline sequence, see market-matching-methodology. For interpreting final outputs, see market-matching-results.

Workflow

Market Matching: Workflow

Workflow at a glance

1. Define test markets and objective
2. Prepare data inputs
3. Submit Market Matching job
4. System computes candidate controls
5. Review match quality outputs
6. Decide to accept, refine, or rerun

---

Step 1: Define the matching objective

Before running:

• identify the test market(s),
• define the outcome metric,
• decide whether controls are auto-selected or explicitly provided.

Helpful question: "Do these controls represent a truly comparable baseline?"

---

Step 2: Prepare data inputs

Minimum expectations:

• clear date field,
• clear location/market field,
• clear outcome metric,
• enough historical coverage for stable pre-period comparison.

Best practice:

• ensure consistent data grain over time,
• remove known data anomalies before matching,
• avoid mixing incompatible market definitions.

---

Step 3: Submit Market Matching job

Jobs are submitted asynchronously:

• request is created,
• background worker executes matching,
• artifacts and metrics are published.

Pipeline options typically include:

• matching method (balanced, causal, quick),
• max controls,
• minimum similarity threshold,
• optional explicit control list.

Input use cases

Market Matching supports three common input modes:

1. Test and control both provided

• You provide test_locations and control_locations.
• System evaluates the provided pair(s) and returns similarity/balance diagnostics.

2. Test provided, control not provided

• You provide test_locations only.
• System auto-selects and ranks the best controls from the remaining candidate markets.

3. Neither test nor control provided

• You provide no explicit markets.
• System auto-selects candidate test markets and controls using historical similarity on the selected outcome metric (for example sales, revenue, or other KPI).

---

Step 4: Automated matching sequence

Behind the scenes:

1. Field mapping and normalization
2. Data transformation and panel balancing
3. Validation checks
4. Geo fingerprint generation
5. Control candidate scoring
6. Balance/quality evaluation
7. Report and metrics publication

---

Step 5: Outputs become available

After completion, review:

• primary match results,
• quality assessment,
• similarity chart,
• treatment-level drilldown,
• PDF report and standardized CSV.

---

Step 6: Decision checkpoint

Choose one path:

• Accept matches: quality is sufficient for downstream use
• Refine setup: adjust test markets, filters, or method
• Rerun: when quality is weak or controls are not credible

---

Pre-run checklist

• Test markets are finalized
• Outcome metric is defined and stable
• Date and location columns are correct
• Sufficient historical window is available
• Similarity threshold strategy is decided

---

Post-run checklist

• Ranked controls reviewed for each test market
• Similarity scores reviewed
• Balance/quality metrics reviewed
• Low-quality or unstable matches flagged
• Go/no-go decision documented

---

Common workflow pitfalls

• Choosing controls only by geography intuition
• Accepting high similarity without checking balance
• Ignoring warning comments in quality output
• Using overly short history windows
• Treating one run as final without sensitivity reruns

Methodology

Market Matching: Methodology

How Market Matching works technically

Market Matching finds the best control markets for a given test market by comparing pre-period behavioral similarity on an outcome metric (sales, revenue, units, etc.).

The process has six stages:

1. Map and transform data
2. Validate and balance the panel
3. Generate geo fingerprints
4. Score and rank control candidates
5. Evaluate balance and quality
6. Publish results and report

---

Stage 1: Data mapping and transformation

Column mapping

Auto-detects:

• date
• location (market/geo)
• outcome (KPI used for similarity — sales, revenue, etc.)

User overrides are supported for outcome, location, and date columns.

Panel balancing

• Standardizes column names
• Aggregates duplicates
• Handles missing data
• Drops geos with excessive missingness

---

Stage 2: Validation

Checks ensure the dataset is ready for matching:

• minimum geos and time periods,
• missingness thresholds,
• structural completeness.

Validation issues are surfaced in the quality output.

---

Stage 3: Geo fingerprinting

Each market gets a behavioral fingerprint from its outcome time series:

Feature	What it captures
mean_outcome	Average level
std_outcome	Volatility
trend	Direction of movement (robust slope)
seasonal_strength	How seasonal the series is
spikiness	Outlier frequency
autocorr	Week-to-week persistence
residual_variance	Noise after detrending
outcome_range	Min–max spread
outcome_cv	Relative variability

These features power similarity scoring between test and control markets.

---

Stage 4: Control candidate scoring

Input use cases

Mode	What you provide	What the system does
Test + control	Both markets specified	Evaluates the pair and returns similarity/balance
Test only	Test market(s) only	Auto-selects and ranks best controls
Neither	No markets specified	Auto-selects test markets and controls by outcome similarity

Ranking

• Candidate controls are scored by similarity to each test market
• Results are filtered by minimum similarity threshold
• Top controls are retained (subject to max control cap)
• An autonomous quality gate can filter poor-quality pairs

---

Stage 5: Balance and quality evaluation

Balance (SMD)

Standardized Mean Difference (SMD) measures how comparable test and control markets are on key features. Lower absolute SMD values indicate better balance.

A pair is considered balanced when SMD stays within the configured threshold across features.

Quality score

Combines:

• similarity strength,
• balance quality,
• and validation issue penalties.

Produces an overall score (0–1) and per-match diagnostics.

---

Stage 6: Output publication

Results include:

• primary matching table (test → ranked controls),
• quality assessment,
• similarity chart data,
• treatment-level drilldown,
• PDF report and standardized CSV.

---

Data requirements

Requirement	Practical guidance
History	Enough weeks for stable fingerprinting
Geos	Multiple candidate markets for meaningful matching
Outcome	Single KPI for similarity (sales, revenue, units, etc.)
Frequency	Weekly time series is typical

---

Matching method options

Three algorithms are available. See market-matching-models for details:

Method	Speed	Depth
balanced	Standard	Full fingerprint set, euclidean similarity
causal	Standard	Propensity-score-based matching
quick	Fast	Core features only (mean, std, trend)

---

What can go wrong

• No similar markets found for the test geo
• High similarity but poor balance (SMD fails)
• Test market dropped during data cleaning
• Quality gate removes all candidates (fallback may salvage top pair)
• Outcome metric changed mid-study

For interpretation guidance, see market-matching-results.

Results

Market Matching: Results and Interpretation

What results you get

A typical Market Matching run provides:

• overall quality score,
• ranked controls for each test market,
• per-control similarity scores,
• balance statistics,
• and report artifacts for review.

---

Read results in this order

1. Overall quality assessment
2. Test market -> control ranking
3. Similarity levels
4. Balance diagnostics
5. Warnings/comments

This order helps prevent selecting controls based only on one metric.

---

Key output sections

Primary results

• One row per test market
• Lists selected control markets
• Includes similarity and balance summary

Quality assessment

• Overall quality score
• Per-test-market quality signals
• Balance audit details

Similarity chart

• Visual ranking of candidate controls
• Helps detect sharp quality drop-offs

Treatment drilldown

• Detailed per-control statistics for a specific test market
• Useful for final control acceptance decisions

---

How to interpret quality levels

High quality

Meaning:

• selected controls are close and stable enough for reliable downstream analysis.

Action:

• proceed with confidence to causal measurement.

Medium quality

Meaning:

• usable controls exist, but match quality is mixed.

Action:

• consider sensitivity rerun (different method/thresholds) before finalizing.

Low quality

Meaning:

• controls are weak or poorly balanced.

Action:

• revise test markets, improve data quality, and rerun.

---

Common diagnostic comments (important)

When quality is weak, read comments before deciding.

Where to read:

• run summary comments in result details
• narrative lines in the generated PDF summary

Comments you may see:

• No similar markets found for selected test market(s)
• Meaning: candidate controls do not match pre-period behavior closely enough.
• Weak similarity / poor balance
• Meaning: controls were found, but comparability is low.
• Insufficient history for stable matching
• Meaning: the pre-period window is too short/noisy for reliable pairing.
• Quality gate fallback applied
• Meaning: strict filters removed all candidates, so best-available fallback pair was kept.

Recommended follow-up:

1. Revisit test market selection
2. Increase historical window
3. Tighten data quality checks
4. Rerun with alternate matching method and compare results

---

Key terms

Similarity score

• Numeric measure of closeness between test and control markets.
• Higher is generally better.

Balance

• How comparable test and control markets are on key behavior features.

SMD (Standardized Mean Difference)

• Scale-free balance diagnostic.
• Lower absolute values indicate better balance.

Quality score

• Composite indicator summarizing match reliability.
• Should be read with warnings/comments, not alone.

---

Decision matrix

Result pattern	Interpretation	Recommended move
High quality + strong balance	Reliable control set	Accept and proceed
Medium quality + mixed balance	Usable but uncertain	Run sensitivity checks
Low quality + weak balance	Unreliable controls	Redesign and rerun
No similar markets found	No credible controls	Re-scope test markets

---

Reporting recommendations

Use this format in reviews:

1. Objective and selected test markets
2. Quality score and key warnings
3. Final control selection rationale
4. Risks and assumptions
5. Next step (accept/refine/rerun)

Avoid:

• selecting controls from similarity alone,
• skipping balance diagnostics,
• ignoring warning comments in low-quality runs.

---

Output surfaces

Typical output surfaces:

• job details summary
• primary results table
• quality assessment panel
• similarity chart and drilldown
• PDF report and downloadable artifacts